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EXECUTIVE SUMMARY 



The No Child Left Behind Act (NCLB) requires districts to offer supplemental 
educational services to students in schools that are in the second year of school 
improvement — that is, schools that have been identified for not meeting the state’s 
adequate yearly progress goals for three consecutive years. NCLB defines supplemental 
educational services as “additional academic instruction designed to increase the 
academic achievement of students in low-performing schools” and states that services 
must “be provided outside the regular school day.” Supplemental service providers may 
include non-profit, for-profit, and faith-based organizations in addition to the public 
schools. These requirements are new and have no precedent in prior federal legislation. 

This report examines the implementation of NCLB ’s supplemental educational service 
provisions in eleven urban districts — Mesa Public Schools and Washington Elementary 
District Schools, AZ, Fresno Unified School District and Los Angeles Unified School 
District, CA, Chicago Public Schools, IL, Buffalo Public Schools and New York City 
Public Schools, NY, Arlington Public Schools and Richmond Public Schools, VA, and 
Atlanta Public Schools and DeKalb County Schools, GA. This geographically, 
politically, and demographically diverse sample of districts provides a range of local 
contexts for studying the implementation of supplemental educational services. Our first 
year study highlights the following findings. 

Supplemental educational services were not widely used during the first year even 
though there were thousands of eligible students, 

• In each of the districts we studied, fewer than 1 8% of eligible students requested 
and received supplemental educational services. In most of these districts it was 
less than 7% of the eligible students, and in some it was less than 1%. 

• This provision disproportionately impacts districts serving large numbers of low 
income and minority students, yet there is little empirical evidence of its 
effectiveness for the most vulnerable students. 

Supplemental educational services placed enormous administrative burdens on 
districts implementing supplemental education services. 

• Districts provided considerable administrative and managerial oversight of the 
development and implementation of the program yet there no additional resources 
to meet these responsibilities. While Title I administrative funds could be used 
for this purpose, districts reallocated these from existing funds and programs. 

• Title I schools also lost resources since districts reduced a school’s Title I 
allocation to cover the costs of supplemental services, diverting resources from 
other reform efforts. This funding strategy limited efforts to plan for long-term 
school reform. 

• Most supplemental educational services programs were not well coordinated with 
the classroom curriculum and there were few mechanisms for providers to 
communicate effectively with classroom teachers. 
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Supplemental educational services are being implemented with little or no 
accountability. 

• Few districts had plans in place to evaluate supplemental services, and those that 
did relied on the most rudimentary evaluation methods. 

• There is no research on how these programs might work or how they might effect 
student achievement and the performance of Title I schools. 

Even though supplemental services were not widely used during the first year, it remains 
important for the following reasons. 

• Supplemental educational services represents a major tenant of the law, that is, 
that competition will produce better educational opportunities for disadvantaged 
students than the public schools provide and that accountability for individual 
student achievement will improve the performance of low performing schools. 
Whether this is true remains untested. 

• The potential for supplemental educational services to fragment Title I is real and 
not addressed in the law. Supplemental services revises the direction of earlier 
Title I legislation that encouraged programs designed to increase curricular and 
instructional integration and diverts resources from the most needy schools. 

• Supplemental educational services shift the focus from improving poorly 
performing schools to improving individual student achievement, but only for 
those requesting services. Combined with the loss of resources, it is unclear how 
this strategy will improve low performing, disadvantaged schools. 

• The demand for supplemental educational services is likely to increase in most 
jurisdictions because the number of students eligible to receive services continues 
to grow and parents seem to prefer supplemental services over the option to 
transfer to another school. 

While disadvantaged students can benefit from access to additional services, how they 
are provided is as important as their availability. Research suggests that stand-alone 
programs tend to fragment the delivery of instruction without any clear educational 
benefit. While there is no panacea to improving the performance of disadvantaged 
students, research has identified some directions that are more effective than others. For 
now, better evidence is needed before public dollars are used to support supplemental 
educational services on a large scale. 
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INTRODUCTION 



Supplemental educational services, along with the public school transfer option, 
exemplify the core principles that underlie the No Child Left Behind Act of 2001 
(NCLB). According to federal policymakers (U.S. Department of Education, 2002a), 

“the choice and supplemental educational services requirements of the law not only help 
to enhance student achievement but also provide an incentive for low-performing schools 
to improve” (p.l 1). Both the transfer option and supplemental educational services are 
based on the assumption that competition will expand the educational opportunities of 
students and create incentives for low-performing schools to improve their instructional 
program. Underlying supplemental services is the assumption that academic instruction 
provided outside the regular school day by public and private organizations will be able 
to do what schools could not — raise the achievement of students in consistently poorly 
performing schools. Supplemental educational services, among the first major provisions 
districts are required to implement, raises questions about the policy process at the district 
level and whether the federal law will improve educational opportunities for 
disadvantaged students. 

While the supplemental educational services provision requires a specific approach to 
improving student achievement, it ignores the capability of mediating organizations, such 
as local educational agencies, to implement it (Timar, 1997). School districts must 
translate the provision into workable programs, mesh it with existing policies, and gain 
support for it among principals, teachers, parents, and the community. The priorities of 
district officials, who are interested in maintaining the legitimacy of the school system, 
include securing resources (money and teachers) to run the system and distributing these 
resources in a way that satisfies the various constituencies with an interest in the school 
system. To the extent that NCLB’s mandates — such as supplemental educational 
services — reinforce those objectives, implementation will proceed smoothly. Where they 
conflict, implementation is likely to be more contentious. 

Supplemental educational services also raise concerns about civil rights and racial equity 
since districts serving large numbers of disadvantaged students are likely to be the first 
required to offer them. Apart from implementation issues, it is unclear whether 
supplemental services will improve the education of low-income and minority students. 
There are no models of this program in existence prior to NCLB and no research on how 
it might work. Furthermore, these services are likely to impose additional burdens on 
districts unless there is a corresponding increase in the resources needed to implement 
them. Supplemental services may weaken the organizational capacity of Title I schools 
to deliver a coherent instructional program by diverting resources away from schools and 
limiting the flexibility of school professionals. By emphasizing accountability for 
individual student achievement, this provision risks leaving behind large numbers of 
students unable to access the additional services. From a civil rights perspective, since 
this remedy for low performance is untested, it is important to understand the potential 
consequences for minority and low-income students. 
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In this paper we examine the ability of distriets to implement the requirement that sehools 
offer supplemental edueational serviees to students attending poorly performing sehools. 
Under NCLB aeeountability, this mandate is among the first interventions to go into 
effeet and is a elear representation of the prineiples diseernable in NCLB — it relies on 
aeeountability and eompetition as mechanisms to improve education and produce better 
opportunities for disadvantaged students. It is also likely to require considerable effort 
from district administrators to translate the provision into programs that can be 
implemented. 

This report is organized as follows. The next section traces the emergence of 
supplemental services as a policy alternative in the NCLB legislation. We examine how 
supplemental services reverses the direction of previous Title I legislation and assess the 
potential benefits and costs of its provisions on the educational opportunities of 
disadvantaged children. Next, we describe the design of the study. In the fourth section, 
we describe student participation in the supplemental educational services program 
during the first year of implementing NCLB (2002-03). We then examine the 
implementation of supplemental educational services, including the administrative and 
management challenges to implementing programs, the costs to districts and local 
schools, and the implications of supplemental services for delivering a coherent 
curriculum. The sixth section discusses the challenges to evaluating the effectiveness of 
supplemental educational services. The final section offers our conclusions and 
rec ommendations . 
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SUPPLEMENTAL EDUCATIONAL SERVICES: DISTRICT 
RESPONSIBILITIES AND CIVIL RIGHTS IMPLICATIONS 



Supplemental edueational serviees emerged as a poliey derivative from the idea of 
“portability,” a eoneept developed in a series of papers published in 1999 by the Fordham 
Foundation (Kanstoroom & Finn, 1999). In one of the papers, Raviteh (1999) argued that 
Title I funding had ereated an “unwieldy bureaueraey” and that “the most direet way to 
reform Title I — and eut its bureaueraey down to size — would be to eonvert it to a 
portable entitlement, available to its intended reeipients for edueational serviees” (p. 

143). The idea that dollars should follow the ehild in the form of a per-pupil alloeation 
was taken up by Senator Judd Gregg (R-NH) in his proposal to re-authorize the 
Elementary and Seeondary Edueation Aet (ESEA) and Title I in 1999-2000. In this 
proposal, the ability to purehase edueational serviees from an instruetional serviee 
provider was part of a larger portability proposal. While the portability proposal was 
defeated, the No Child Left Behind Aet of 2001 (NCLB) retained the idea of 
supplemental edueational serviees as one of the sehool improvement alternatives 
available to students in low performing sehools. The idea was not based on previous 
experienee or researeh but represented a politieal eompromise between supporters and 
opponents of vouehers. 

NCLB requires distriets to offer supplemental edueational serviees to students in sehools 
that are in the seeond year of sehool improvement — that is, sehools that have been 
identified for not meeting the state’s adequate yearly progress goals for three eonseeutive 
years. The federal law permits distriets to offer supplemental serviees to students in 
sehools that were in their first year of sehool improvement if sehool ehoiee was not 
possible.^ As a result, many parents whose ehildren were eligible to transfer to another 
sehool eould ehoose instead to reeeive supplemental serviees. NCLB defines 
supplemental edueational serviees as “additional aeademie instruetion designed to 
inerease the aeademie aehievement of students in low-performing sehools,” and indieates 
that they “must be high quality, researeh-based, and speeifieally designed to inerease 
student aehievement.” The federal legislation also requires that supplemental serviees “be 
provided outside the regular sehool day,” whieh may inelude after-sehool and weekend 
programs (U.S. Department of Edueation, 2002, Deeember 12) [P.L. 107-110, See. 
1116(e)(12)(C)] 

Under NCLB, the supplemental serviee requirements are new and have no preeedent in 
prior federal legislation. Unlike sanetions imposed in the past, they are no longer 
optional or at the diseretion of distriet offieials. In addition to the publie sehools, 
supplemental serviee providers may inelude non-profit, for-profit, and faith-based 
organizations, eharter sehools, private sehools, and publie and private eolleges and 
universities (U.S. Department of Edueation, 2003, August 22). While this requirement 
potentially inereases the supply of providers, it plaees additional monitoring and 



'According to the federal statute, districts must provide all students enrolled in a school that is in its first 
year of school improvement (schools identified for not meeting the state’s adequate yearly progress goals 
for two consecutive years) “with the option to transfer to another public school served by the local 
educational agency” (P. L. 107-110, § 1116(b)(l)(E)(i)). 
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bureaucratic duties on the district. The district must enter into agreements with all 
providers and monitor their progress towards improving student achievement. In 
addition, districts are required to set achievement goals, develop a plan for monitoring 
student progress, and outline a timetable for improving student achievement for students 
requesting supplemental services (P.L. 107-110, Sec. 1116(e)(3)(A)). Districts must 
inform parents about the availability of these services as well as provide them with 
reports on student progress. Finally, districts are required to set aside a portion of their 
Title I allocation to pay for supplemental services and to make all arrangements to pay 
the providers. 

These supplemental service requirements reverse the direction of earlier Title I legislation 
that moved the program in the direction of establishing schoolwide programs coordinated 
with the regular curriculum. One of the criticisms of the early categorical Title I program 
was that the delivery of instruction was characterized by curricular and instructional 
fragmentation and that there was little coordination between the Title I program and the 
regular curriculum (Jeffrey, 1978; Johnston, Allington, & Walker, 1985; Kaestle & 

Smith, 1982; Kirst, 1988; Kirst & Jung, 1982; Martin & McClure, 1969). Recognizing 
this as a problem, federal officials began to soften requirements that program services be 
distinct and easily identifiable in several ways. First the 1988 Hawkins- Stafford 
Amendments to the ESEA and later the 1994 Improving America’s Schools Act gave 
local school districts and schools greater flexibility to decide where and how to use the 
federal Title I resources and encouraged the adoption of schoolwide programs. To 
further encourage the adoption of schoolwide programs, the federal law successively 
lowered the poverty cutoff point required for schools to qualify for adopting a schoolwide 
program. While not a panacea, schoolwide programs eliminated some of the major 
obstacles to integrating Title I services with the school curriculum. 

The earlier Title I legislation also granted increased flexibility to school professionals to 
address the concentration effects of disadvantaged students in poor neighborhoods 
(Wong, Sunderman, & Lee, 1997). It directed additional resources to schools serving 
disadvantaged students and promoted flexibility in the use of those resources to 
encourage instructional innovation and coordination between the Title I program and the 
regular curricular program. Einally, prior ESEA legislation incorporated accountability 
by requiring that the same standards apply to all students, including those in high-poverty 
Title I schools. Taken together, the idea was that these approaches would foster deep and 
comprehensive school reform that would demonstrate results. 

As subsequent sections of this report will show, these approaches to a coordinated and 
comprehensive reform of Title I schools are threatened under NCLB. NCLB fails to 
provide a clear understanding of whether supplemental educational services will 
contribute to the development of coherent instructional programs or whether service 
providers will be compelled to coordinate their services with classroom teachers or 
school principals. Instead, the supplemental service provisions weaken the organizational 
capacity of schools to develop a coherent instructional program. By directing resources 
to outside service providers, it reverses earlier attempts to provide additional resources to 
needy schools and limits the school’s ability to develop comprehensive strategies to help 
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disadvantaged students. In faet it reduees a sehool’s Title I alloeation sinee Title I funds 
must be “set aside” to pay the providers. And, it deereases aeeountability by emphasizing 
short-term aeeountability for individual student aehievement. Rather than a foeus on a 
broad range of school level outcomes tied to state standards and the development of 
school improvement plans to meet those standards, it focuses on improving individual 
student achievement, but only for those requesting services. 

Advocacy organizations remain divided over the potential benefits and costs of 
supplemental services on minority children’s learning opportunities and outcomes. On 
one hand, some advocates believe that the expansion of supplemental education services 
has the potential to equalize learning opportunities between White and minority students. 
This view was recently echoed in a White House Report on the education of Hispanic 
children (President's Advisory Commission on Educational Excellence for Hispanic 
Americans, 2003). Among the policy recommendations, it stressed that “Hispanic 
American parents with children in poor-performing public schools must have a legitimate 
opportunity to exercise these options under NCEB” (p. 6). Other organizations, however, 
have voiced strong opposition to NCEB’s supplemental service provisions, viewing it as 
a thinly veiled attempt to “voucherize” public education. Eor example, a leader of the 
National Alliance of Black School Educators (NABSE) asserted in a recent editorial: 

The Alliance considers educational vouchers as a "bad" quick-fix solution 
that addresses the needs of only a few children while ignoring the 
continuing educational plight of the majority of poor children. It is morally 
dishonest to take attention away from creating structures and resources to 
assure quality education for poor children by directing attention and 
resources through vouchers to a symbolic "lucky few.” (Gray, 2003) 

Thus far, research provides little evidence to guide policymakers and advocates who 
disagree about the benefits of supplemental education services. The most rigorous 
national evaluation of the 2E‘ Century Teaming Community Centers, an after-school 
program for children in urban and mral communities, has shown the program had limited 
effects on student achievement and modest impacts on some non-cognitive indicators 
(U.S. Department of Education, 2002b).^ Moreover, summer school programs, which 
have been developed in response to research showing that the achievement gap forms and 
widens primarily during summer vacation (Entwisle, Alexander, & Olson, 1997; Heyns, 
1978; Murnane, 1975), failed to narrow the achievement gap between low- and middle- 
income students (Cooper, Charlton, Valentine, & Muhlenbmck, 2000). As a result, some 
scholars (Entwisle, Alexander, & Olson, 2000) have suggested that if “summer school 
and other programs [are] to close the learning gap, they have to be designed especially for 
poor children and provided only for them” (p. 25). 

While many programs have shown limited success, research does point to some 
promising approaches that have been beneficial for improving the achievement of 
disadvantaged students. Research on the implementation of Title I suggests that all 



^ Mathematica conducted a national evaluation of 2E‘ Century Learning Centers. Impact was greater only 
when there was parental involvement. 
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students benefit in sehools that adopt a comprehensive approach to educating children. 
This includes adopting a Title I curriculum that is coordinated with the regular 
curriculum, providing programs that support instruction students receive in the core 
curriculum, and developing expectations that are the same for all students (Orfield & 
DeBray, 1999; Wong et ah, 1997). Programs that focused on individual student 
remediation that were not coordinated with the regular classroom curriculum were less 
successful (Wong et ah, 1997). Other approaches that have a strong record of improving 
learning outcomes for minority and low-income students rely on the flexibility of school 
professionals to use resources to address the effects of concentrated poverty. For 
example, research on Title I shows that achievement gains are tied to instructional 
interventions, such as qualified teachers and class-size reduction, targeted at high- 
poverty, high-minority schools (Grissmer & Flanagan, 1999; Orfield & DeBray, 1999). 
Reducing class sizes in the early grades has potential to narrow the racial achievement 
gap since it produces larger achievement benefits for minority students than White 
students (Finn & Achilles, 1999; Krueger, 1999; Nye, Hedges, & Konstantopoulos, 
2000). Since Title I funds often support these instructional interventions, diverting funds 
to other purposes may impact the ability of schools to implement these and other 
instructional programs that benefit disadvantaged students. 
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DESIGN OF STUDY 



Research Questions on Supplemental Educational Services 

To begin to understand how distriets and schools implemented supplemental educational 
services in the first year of implementing NCLB (2002-03), we examined three questions. 

1 . What was the extent of participation in supplemental educational services during 
the first year of implementation? 

2. How did districts implement supplemental educational services in the first year? 

3. What are the challenges to effectively evaluating supplemental educational 
services? 

District Selection Criteria and Characteristics 

We purposefully selected 1 1 districts that enrolled large numbers of minority and low- 
income students. As shown in Table 1, our sample is diverse with respect to geography 
and size. Each district is located in one of the six states that are part of our national study 
on NCLB, including Arizona, California, Illinois, New York, Virginia, and Georgia. The 
sample includes the nation’s three largest public schools districts: Los Angeles Unified 
School District, the Chicago Public Schools, and the New York City Public Schools. 
Together, these three districts enroll over 2 million students in 1,807 schools. Three 
districts — Mesa, AZ, Lresno, CA, and DeKalb County, GA — are among the nation’s 50 
largest school districts (Sable & Young, 2003). The five remaining districts are located 
in the “central-city” portion of the Metropolitan Statistical Area (MSA) in Phoenix, AZ 
(Washington Elementary), Buffalo, NY, Washington, DC (Arlington County, VA), 
Richmond, VA, and Atlanta, GA. 

These districts enroll a large percentage of low-income and minority students. Minority 
students make up over 90% of the total enrollment in Los Angeles, Chicago, Richmond, 
and Atlanta, and over 80% in Lresno, New York City, and DeKalb County (Table 1). 
Buffalo enrolls 72% minority students. Over half of all students in these districts receive 
a federal meal subsidy. The two Arizona districts (Mesa and Washington) and Arlington 
County, Virginia have comparatively lower poverty rates and a smaller proportion of 
minority students. But since they do have a diverse student population, they are 
representative of many districts across the nation undergoing racial and socio-economic 
changes in K-12 enrollment. 
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Table 1: Total Enrollment and Percentage of Minority and Low-Income Students 
in 11 District Sample, 2001-02, 



District 


Total 

Enrollment 


% Minority 


% Low-Income* 


Mesa Unified, AZ** 


74,808 


36 


36 


Washington Elementary Distriet, AZ 


24,811 


42 


49 


Fresno Unified, CA 


81,058 


81 


75 


Los Angeles Unified, CA 


735,058 


90 


73 


City of Chieago Publie Sehools, IE 


437,418 


91 


84 


Buffalo Publie Sehools, NY 


44,849 


72 


82 


New York City Publie Sehools, NY 


1,049,831 


85 


76 


Arlington County Publie Sehools, VA 


19,109 


58 


41 


Riehmond City Publie Sehools, VA 


24,840 


93 


64 


Atlanta Publie Sehools, GA 


56,586 


93 


80 


DeKalb County Sehool Distriet, GA 


97,501 


89 


56 



Source: National Center for Education Statistics, Common Core Data, http://nces.ed.gov/ccd/districtsearch/ 
*We defined "low-income" as the percentage of students receiving free- and reduced-price lunch. 

**Data on free- and reduced price lunch for Mesa was provided by the district. 



When we examine particular district characteristics, including student performance, we 
find that students in these districts usually perform below the state’s average 
performance. Table 2 compares the state and district proficiency rates in reading and 
math in the lowest tested grade, that is, the percentage of students who score at or above 
the state defined proficiency level. In most cases, the average reading and math 
proficiency rate for each district is lower than the state average. For example, the 
average fourth-grade reading proficiency rate in Fresno (27%) and Los Angeles (21%) 
was below the average for California (36%). The differences between the average district 
proficiency rate and state proficiency rates were smaller in the two Georgia districts and 
in Arlington. 
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Table 2: State and District Proficiency Rates in Reading and Math, 
Lowest Tested Grade, Test Administered in Spring 2002, 



District 


Reading 
% Proficient 


Math 

% Proficient 


Arizona (Grade 3) 


74 


80 


Mesa 


83 


57 


Washington 


75 


44 


California (Grade 4) 


36 


37 


Fresno 


27 


30 


Los Angeles 


21 


34 


Illinois (Grade 5) 


59 


63 


Chicago 


37 


36 


New York (Grade 4) 


64 


70 


Buffalo 


36 


50 


New York City 


52 


67 


Virginia (Grade 3) 


72 


80 


Arlington 


75 


84 


Richmond 


54 


60 


Georgia (Grade 4) 


77 


66 


Atlanta 


71 


56 


DeKalb 


76 


64 



Source: We obtained achievement score results from the following: 
http://accountabilitv.doe.kl2.ga.us/Report02/ (Atlanta, Georgia CRCT) 
http ://2Q6. 166. 105. 128/ReportCard/rchome. asp (Chicago, Illinois ISAT) 
http://star.cde.ca.gOv/star2002/district index.html (Los Angeles, California CST) 
http://www.nvcenet.edu/daa/2003ELA/ (New York, NY ELA4) 
http ://www.pen.kl 2.va.us/VDOE/ src/ (Arlington, Richmond, Virginia SOL) 
http://www.ade.az.gov/srcs/ (Mesa, Washington, Arizona, AIMS) 



Finally, a disproportionately large percentage of the schools identified as needing 
improvement in each state were in the districts in our sample. Under NCLB, schools are 
identified for improvement it they do not meet the state’s adequate yearly progress goals. 
The schools are then required to implement a series of sanctions, ranging from public 
school transfers, to supplemental educational services, corrective action, and 
restructuring, depending on how many years the school failed to make adequate yearly 
progress. Table 3 compares each district’s share of all schools in the state with the 
district’s share of schools identified for improvement. These schools are in the first year 
of school improvement, which means they had not meet the state’s adequate yearly 
progress goals for two years. A sub-sample of this group of schools would be required to 
offer supplemental services. For example, Fresno Unified contains only 1% of the 
schools in California, but nearly 5% of the schools in the state that were identified as 
needing improvement. Los Angeles contains 7% of California’s public schools, but 
nearly 13% of the state’s improvement schools. Chicago, New York City, and Richmond 
contain at least half of the schools identified for improvement in their respective states. 
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Table 3: Number of Improvement Schools by District, and the Percentage of 
Improvement Schools in Each District as Percentage of State Total, 2002-03. 





Schools 


Improvement Schools 


District 


Number 


Percentage of 
State Total 


Number 


Percentage of 
State Total 


Arizona 


1,803 




399 




Mesa 


88 


4.9 


18 


4.5 


Washington 


25 


1.4 


6 


1.5 


California 


8,916 




814 




Fresno 


99 


1.1 


39 


4.8 


Los Angeles 


663 


lA 


106 


13.0 


Illinois 


4,351 




527 




Chieago 


599 


13.8 


335 


63.6 


New York 


4,296 




434 




Buffalo 


76 


1.8 


31 


7.1 


New York City 


1,164 


27.1 


382 


88.0 


Virginia 


2,090 




34 




Arlington 


32 


1.5 


0 


0.0 


Riehmond 


63 


3.0 


17 


50.0 


Georgia 


1,969 




436 




Atlanta 


99 


2.8 


28 


6.4 


DeKalb 


129 


6.6 


25 


5.7 



Source: See Appendix 1 for data sources. 



Data Collection Methods 

We used both qualitative and quantitative sources of data for this study. To examine how 
districts implemented supplemental educational services, we conducted interviews with 
district officials responsible for implementing the program as well as superintendents, 
associate superintendents. Title I program coordinators, human resource directors, and 
transportation coordinators. In addition to interview data, we reviewed district 
documents and policies related to supplemental educational services. We augmented our 
interview data with accounts from newspapers and district press releases. Second, we 
collected district statistics on the number of schools identified for improvement and the 
number of students requesting and taking advantage of supplemental services. We 
constructed a database with data from all public schools in each state, including Title I 
program status (schoolwide vs. targeted assistance), number of years in school 
improvement, enrollment data, student demographic characteristics, and achievement 
outcomes. Data collection took place between October 2002 and July 2003. 

In many districts, the number of schools identified for improvement in 2002-03 changed 
as states renegotiated adequate yearly progress rules with the federal government and as 
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districts found errors in the list of sehools the state had identified for improvement. This 
meant that the number of sehools that had to offer supplemental serviees ehanged during 
the 2002-03 sehool year in many distriets. In eondueting our analyses, we reeoneiled the 
differenees in two ways. First, our list of sehools identified for improvement is eurrent as 
of June 2003, whieh marks the end of the 2002-03 sehool year in most distriets. Seeond, 
we used multiple sourees to verify our lists of improvement sehools, ineluding data 
obtained through distriet doeuments, interviews with distriet Title I direetors, and 
national and regional newspapers. 
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PARTICIPATION IN SUPPLEMENTAL EDUCATIONAL SERVICES 



Students were eligible for supplemental serviees if they attended a sehool that was in its 
second year of school improvement — that is, the school had not met the state’s adequate 
yearly progress goals for three consecutive years. The state educational agency has the 
responsibility to identify providers, maintain a list of providers, and monitor services 
while the district must notify parents about the availability of service, arrange for services 
to be provided, and also monitor the providers to be sure student achievement has 
improved. Parents are responsible for deciding whether or not they want their child to 
receive supplemental educational services and for picking a service provider. Providers 
include both public and private organizations and services must be provided outside of 
the regular school day. To pay for the services, NCLB requires districts to set aside a 
portion of their Title I allocation and places a cap on the amount that can be spent on each 
student. The district may spend less than this amount if demand for services is low, but 
must hold this money in reserve until a program is in place. 

Our analysis of student participation during the first year of NCLB examines student 
eligibility for services and disaggregates eligibility by race and ethnicity. Eligible 
students were disproportionately minority students. While large numbers of students were 
eligible for services, those who actually took advantage of the program was quite small. 
These results are presented in detail below. 

Student Eligibility for Supplemental Educational Services, by Race and Ethnicity 

As mentioned earlier, students are eligible to receive supplemental educational services if 
they attend a school that fails to make adequate yearly progress for three or more years 
(second year of school improvement). Since one of our districts — ^Arlington — ^was not 
required to offer supplemental services in 2002-03, our analysis focuses on 10 districts. 
Table 4 shows the number of schools in each district that had to offer supplemental 
services to their students. The percentage of schools required to offer supplemental 
services ranged from 4.1% in Chicago to 28.3% in Atlanta. Table 5 shows the number of 
eligible students. Although the number of eligible students varied across the 10 districts, 
there are literally thousands of students who were eligible to receive services in each 
district. In the nation’s three largest districts, the number of students eligible for 
supplemental services ranged from over 240,000 in New York City, to 164,000 in Los 
Angeles, and over 17,000 in Chicago. Fresno, a much smaller district than Chicago, also 
had almost 17,000 eligible students. As a percent of total enrollment, eligibility ranged 
from a low of 4% in Chicago to a high of 24% in Richmond. In most districts, about a 
fifth to a quarter of the students were eligible for services. 
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Table 4; Number and Percentage of Schools Required to Offer 



Supplemental Educational Services, 2002-03. 



District 


Number of 
Schools 


Schools 
Required to 
Offer Services 


Percentage of 
Total District 
Schools 


Mesa, AZ 


88 


10 


11.4 


Washington, AZ 


25 


6 


24.0 


Fresno, CA 


99 


21 


21.2 


Los Angeles, CA 


663 


106 


16.0 


Chicago, IL 


599 


25 


4.2 


Buffalo, NY 


76 


18 


23.7 


New York, NY 


1,164 


280 


24.1 


Richmond, VA 


63 


17 


27.0 


Atlanta, GA 


99 


28 


28.3 


DeKalb, GA 


129 


17 


13.2 



Source: See Appendix 1 . 



Table 5: Eligibility for Supplemental Educational Services as a 
Percent of Total Enrollment, 2002-03. 



District 


Enrollment 


# of Eligible 
Students 


% of Total 
Enrollment 


Mesa, AZ 


74,808 


6,143 


8.2 


Washington, AZ 


24,811 


3,314 


13.4 


Fresno, CA 


81,058 


16,831 


20.8 


Los Angeles, CA 


735,058 


164,434 


22.4 


Chicago, IL 


437,418 


17,455 


4.0 


Buffalo, NY 


44,849 


9,196 


20.5 


New York, NY 


1,049,831 


243,249 


23.2 


Richmond, VA 


24,840 


6,033 


24.3 


Atlanta, GA 


56,586 


13,448 


23.8 


DeKalb, GA 


97,501 


9,355 


9.6 



Source: See Appendix 1 . 



Many of these eligible students were minorities. In Eigure 1, we compare the 
racial/ethnic makeup of students in schools offering supplemental services in the 10 
districts. The bar graph clearly shows that the vast majority of students in these schools 
were minority students, including Eatino students in the four western districts (Mesa, 
Washington, Eresno, Eos Angeles), Black and Eatino students in Chicago and New York 
City, and mainly Black students in the three southern districts (Richmond, Atlanta, 
DeKalb) and Buffalo. In our two California districts, Eresno and Eos Angeles, over 50% 
of the students in schools required to offer supplemental services were Eatino. In the 
three districts located in the southeastern region (Richmond, Atlanta, DeKalb), over 75% 
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of students were Blaek. In Chicago and in New York City substantial numbers of both 
Black and Latino students were eligible for services. 



Figure 1 ; Racial/Ethnic Makeup of Schools Offering Supplemental Educational Services 
in 10 School Districts, 2002-03. 




□ Asian 

□ Black 

□ Latino 

□ White 



District 



Source.' See Appendix 1 . 

Student Participation in Supplemental Educational Services 

While large numbers of students were eligible to receive supplemental services, the 
number requesting and receiving services was extremely low, as shown in Table 6. The 
percentage of students who took advantage of supplemental services ranged from less 
than 1% in Mesa, Washington, and Eresno to about 18% of the eligible students in 
Atlanta. Indeed, in Mesa there were only two students who requested and received 
services. In the other districts in our study, those receiving supplemental services ranged 
between 2% in Richmond to 12.5% in New York City. About 6% of eligible students 
received services in Eos Angeles, Buffalo, and DeKalb and about 5% in Chicago 
received services. While it is difficult to account for the low response rate or the 
differences in response rate between districts, the low response meant that districts could 
more easily accommodate those students requesting services. But it also suggests that 
students in low-income, high minority districts may not take advantage of supplemental 
educational services simply because the services are offered outside of regular school 
hours and away from their school building. 
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Table 6: Student Participation in Supplemental Educational Services by District, 2002-03 . 



District 


Eligible Students 


Requested Supp. Ed. 


Received Supp. Ed. 




# 


# 


% of eligible 


# 


% of eligible 


Mesa, AZ 


6,143 


2 


0.0 


2 


0.0 


Washington, AZ 


3.314 


20 


0.6 


20 


0.6 


Fresno, CA 


16,831 


234 


1.4 


36 


0.2 


Los Angeles 


164,434 


11,518 


7.0 


10,247 


6.2 


Chicago, IL 


17,455 


1,400 


8.0 


850 


4.9 


Buffalo, NY 


9,196 


849 


9.2 


573 


6.2 


New York, NY 


243,249 


30,349 


12.5 


30,349 


12.5 


Richmond, VA 


6,033 


600 


9.9 


122 


2.0 


Atlanta, GA 


13,448 


2,380 


17.7 


2,380 


17.7 


DeKalb, GA 


9,355 


575 


6.1 


575 


6.1 



Source: See Appendix 2. 



Although only a tiny fraction of eligible students requested and received supplemental 
services, this figure was still higher than the number of students who participated in the 
public school transfer program. Table 7 compares the number of students who requested 
supplemental services with the number of students who requested a transfer to another 
school. In most districts, more parents requested supplemental services than they did a 
transfer. For example, 849 of the eligible students in Buffalo requested tutoring whereas 
only 79 requested a transferred to another school; in New York City, over 30,000 
requested supplemental services compared to 6,400 requests for transfers.^ Only in 
Chicago were there more requests for transfers (2,401) than for supplemental services 
(1,400). 



^ See our report on choice (Kim & Sunderman, 2004) for more information on the transfer program under 
NCLB and for data on the number of parents requesting transfers. 
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Table 7: Comparison of Student Requests for Supplemental 
Educational Services and for Transfers by District, 2002-03. 



District 


# of Eligible Students 
Requesting Supp. Ed. 


# of Eligible Students 
Requesting Transfers 


Mesa, AZ 


2 


0 


Washington, AZ 


20 


0 


Fresno, CA 


234 


183 


Los Angeles, CA 


11,518 


N/A 


Chicago, IL 


1,400 


2,401 


Buffalo, NY 


849 


79 


New York, NY 


30,359 


6,400 


Richmond, VA 


600 


123 


Atlanta, GA 


2,380 


32 


DeKalb, GA 


575 


49 



Source: See Appendix 2 for information on the number of students requesting supplemental services. 
N/A: Los Angeles did not implement a NCLB transfer policy in 2002-03. 
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IMPLEMENTING SUPPLEMENTAL EDUCATIONAL SERVICES 



It is too early in the proeess to examine the effect of supplemental services on student 
achievement. During the first year of NCLB (2002-03), districts struggled just to put 
programs in place. Most districts did not receive the list of schools that would be 
required to offer services until after the start of the school year and final guidance from 
the U.S. Department of Education was not released until December 2002. Consequently, 
there was some ambiguity whether schools would be required to offer services in the first 
year. While draft guidance was available in August, some significant issues were not 
clarified until December. Two important pieces of information were new. First, the 
December guidance urged districts to offer supplemental services to schools in their first 
year of school improvement when transfers were not available. Second, it clarified that 
schools identified for improvement after the beginning of the 2002-03 school year were 
required to offer supplemental educational services immediately. The late date of this 
requirement meant that most districts were not prepared to offer programs. Additionally, 
once districts did offer the program, many providers did not participate. Some providers 
wanted a guaranteed minimum number of students before they would offer service and 
would not serve a small number of students. Others did not participate because they 
considered the program too short to justify their setup costs. 

Our analysis of the implementation of supplemental educational services during the first 
year revealed that NCLB shifts responsibility for providing additional educational 
services from Title I schools to the district. Our analysis highlights three findings. First, 
the NCLB requirement to offer supplemental educational services significantly increased 
the administrative and bureaucratic burden on district officials by requiring them to 
develop, implement, and monitor the program. The short implementation timeline, 
unclear regulations, and start-up challenges added to the administrative burden during the 
first year. Second, the set aside requirement, the amount of Title I money districts were 
required to set aside for supplemental services, was a financial burden on schools 
providing Title I services and created incentives for districts to provide their own 
services. Third, it was difficult for schools to coordinate the provision of supplemental 
services with classroom instruction and other programs designed to help low performing 
students. 

Increased Administrative Burden 

Implementing supplemental services received a lot less media and public attention than 
the NCLB transfer option, but in many ways it is much more challenging and potentially 
costly to districts. There are immense administrative and management challenges to 
implementing a supplemental service program. Districts are required to notify parents 
about the availability of supplemental services, process applications, and negotiate an 
agreement with each provider selected by students. They also must monitor the providers 
for accountability purposes to ensure student achievement improves and provide 
information the state needs to monitor the quality and effectiveness of the providers. In 
addition to notifying parents about the availability of services, districts must provide 
parents with specific information about the services, qualifications, and evidence of 
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effectiveness of each provider. Administratively, districts must negotiate contracts with 
each provider that includes specific district expectations and payment arrangements. In 
most districts, the school board must approve each contract. Additionally, administrative 
costs incurred in providing supplemental educational services are not provided for in the 
law. Districts may use their Title I funds to cover administrative costs, but this becomes 
a reallocation of existing funds rather than additional resources for a new mandate. 

The number of approved providers for each district is shown in Table 8. In each district 
there was a mix of local and national providers and Internet providers; seven of the ten 
districts were also service providers. Since the list of potential providers can change 
from year to year, either because the state adds new providers or providers drop off the 
list, districts will have to continually develop new service contracts. When we conducted 
our fieldwork, none of the districts had established how they would monitor the providers 
or the effectiveness of the program. They were also vague about how communication 
between providers and the regular classroom instructors would be maintained. 



Table 8: Number of Approved Supplemental Educational Service Providers Statewide 
and by District, 2002-03. 



District 


Approved Providers 
Statewide 


Approved 
Providers District 


Active Providers 
First Year 


District Provides 
Services 


Mesa, AZ 


26 


11 


1 


No 


Washington, AZ 


26 


9 


1 


Yes 


Fresno, CA 


120 


6 


1 


No 


Los Angeles, CA 


120 


23 


21 


Yes 


Chieago, IL 


18 


18 


5 


Yes 


Buffalo, NY 


104 


17 


6 


Yes 


New York, NY 


104 


47 


40 


Yes 


Riehmond, VA 


21 


16 


2 


No 


Atlanta, GA* 


14 


6 


N/A 


Yes 


DeKalb, GA 


14 


2 


2 


Yes 



Source: Civil Rights Project interviews with district Title I directors. N/A: Not available. 

*The Georgia State Board of Education approved 14 providers statewide that could serve students in any 
district in the state. The number of approved providers reflects providers approved to serve the Atlanta and 
DeKalb County districts. There were 144 local providers statewide, many of them local school districts. 

The program was difficult for district administrators to manage and required considerable 
time on their part. There were numerous administrative procedures districts had to 
develop to set-up a supplemental services program. For one, they had to find ways to 
account for attrition from the program. Since both mobility and student absenteeism are 
chronic issues in many urban districts, student attendance at after school or Saturday 
programs could not be counted on. From the district perspective, administrators wanted 
to be sure that if they were paying a provider, there was a student receiving services. 
District officials also struggled with determining the location and schedule of services. 
They had to balance the reality that if the services were far from the home school or 
inconvenient, students would not attend. If a district was a service provider, union 
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contracts influenced how districts could develop a program, ineluding which teachers 
could apply to teach in the program and building utilization for after-sehool or Saturday 
programs. Districts, which are ultimately responsible for student achievement, also had 
to develop new data tracking and reporting systems to monitor the quality and 
effectiveness of the services offered by providers. Issues about sharing students’ 
confidential information with providers had to be addressed. 

Since NCLB did not provide additional resources for the administration of the program, 
most districts shifted this responsibility onto existing staff. For instance, in Los Angeles 
Title I staff members worked overtime to create brochures, databases, applieation and 
enrollment proeedures, developed special transportation plans for Saturday programs, 
monitored attendance, and supervised the serviees provided by the district. A few 
distriets hired additional staff. Perhaps underscoring the additional bureaucratic 
responsibilities, the only new hire during the 2002-03 school year within the distriet 
offiee in Buffalo, a distriet with a large budget deficit, dealt solely with the administration 
of the supplemental services program. DeKalb hired site eoordinators to help manage the 
program at the sehool site and to faeilitate the transportation of students from their home 
sehool to the site where services were offered. Aecording to the Title I Director in 
DeKalb, “there is going to be a site coordinator so that teachers can focus on the tutoring 
part of it and any other things that come up, the site coordinator will take eare of’ (district 
official, personal communication with G. Sunderman, February 14, 2003). In addition to 
staff that set up and managed the program, districts involved legal counsel, transportation 
staff, human resources, eommunications, and budget office personnel in various aspeets 
of implementing the program. For example, legal counsel reviewed service provider 
contraets, transportation staff developed busing schedules, and human resources vetted 
the personnel that provided the tutoring. 

During the first year, implementation was eonfounded by the short implementation 
timeline, unclear regulations, and start-up ehallenges. Draft guidance, released in 
Deeember 2002, required sehools to offer supplemental services at the beginning of the 
2002-03 school year, even if the sehool was designated as not making adequate yearly 
progress after school had already started (U.S. Department of Edueation, 2002, Deeember 
12). It also encouraged districts to consider offering the program to students who were 
unable to take advantage of the transfer option. This meant many districts and schools 
that thought they would not have to offer serviees had to reconsider. Many distriets did 
not receive a list of potential providers until after the start of the sehool year and then had 
to negotiate the terms of the contraet with the service providers. Start-up time, even for 
small programs, took longer than expeeted. For instanee, Richmond officials had 
estimated that it would take them two to three weeks to set up a program, while it aetually 
took six to eight weeks (district official, personal communication with G. Sunderman, 
May 27, 2003). The proeess was delayed in part by the time serviee providers needed to 
put programs in plaee. 

Nonetheless, most districts offered various kinds of supplemental service programs, most 
starting later in the sehool year. For example, the Chicago Public Schools offered a six- 
week program beginning in April 2003. The program in Richmond evolved as distriet 
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officials received more information from the federal government on the requirements 
governing supplemental serviees. They provided two opportunities for students to enroll 
in supplemental serviees — students who had submitted applieations to transfer to another 
school but were not moved were offered supplemental serviees in the fall of 2002; 
otherwise, eligible students eould enroll in a six-week program beginning in April 2003. 
The programs in Mesa, Buffalo, and at Washington Elementary began in winter 2003 and 
eontinued through the summer, thus providing students with extended aeeess to 
supplemental serviees. Fresno offered a program beginning in February 2003. In Fos 
Angeles, where the sehool board approved the serviee eontraets on Mareh 25, 2003, the 
program began after that and eontinued through August 2003. 

Many serviee providers, partieularly national ones, did not partieipate in district programs 
in the first year even though they were on the state approved list (Table 8). They were 
eautious about entering new markets and sought to proteet themselves against finaneial 
loss. For example, in Richmond, there were 16 providers but only two that participated 
in the distriet’s six -week spring program. Many of the national providers refused to 
provide serviees during the spring 2003 session unless they had a minimum number of 
students that justified setting up in the distriet. Others felt six weeks was too short to 
justify their eosts. In Chieago, only five out of 18 vendors provided serviees. Some 
providers made demands on distriets that were difficult or impossible to meet, including 
guaranteed enrollment figures and permission to use sehool faeilities to reduee their 
eosts. This was the ease in Fresno, where Sylvan, a national vendor, wanted to hire 
district teachers and use district facilities. Aecording to the district Title I Direetor, “they 
wanted to hire our teachers and pay them $25 an hour . . . they were going to provide 
serviee in groups of 6 and wanted us to pay them $27 an hour for trials. So we would pay 
them 6 times $27 for our own teaehers, our own elassrooms — it was just pure profit” 
(distriet offieial, personal oommunieation with J. Jellison Holme, July 31, 2003). In 
Riehmond, one vendor required volunteers to run their program, and while the eompany 
trained the volunteers, the distriet was responsible for reeruiting and monitoring the 
volunteers. 

While some of the difficulties of implementing supplemental educational services were 
related to first year start-up ehallenges, the program is likely to eontinue to require 
extensive administrative support and management. The number of eligible students will 
inerease as more schools are identified for improvement, requiring distriets to process 
more applications and, if requests exeeed available funds, review student performanee in 
order to give priority to the lowest aehieving students. The list of providers is likely to 
ehange, requiring new serviee eontraets eaeh year that include payment arrangements and 
specifie aehievement goals for eaeh student reeeiving serviees. The evaluation 
eomponent is formidable, something we discuss in another seetion. 

Costs of Supplemental Educational Services 

NCFB requires distriets to set aside 20% of their Title I, Part A alloeation for ehoiee- 
related transportation and supplemental edueational serviees (Table 9). Distriets must 
spend at least 5% of this set aside on supplemental edueational serviees unless demand is 



25 




less than estimated. This set aside cannot be used to cover administrative costs of 
managing the program. The per-child cost for supplemental educational services is 
determined by dividing the district’s Title I, Part A allocation by the number of children 
residing within the district aged 5-17 who are from families living below the poverty 
level (U.S. Department of Education, 2003, August 22)."^ In the 2002-03 school year, 
districts could not use the set aside, which was substantial, until after a supplemental 
service program was in place. While districts could reallocate any unused funds, this 
money sat unused until late in the school year because most districts did not begin 
offering a program until January 2003 or later. In Buffalo, a district facing severe 
financial strain, the set aside meant that some schools were unable to provide tutoring in 
the fall. According to the superintendent: 

One of the things that has been frustrating to us — majorly frustrating — is 
that Supplemental Services piece because typically we would have been 
using that money at the end of September. We would have had our thing. 

We would have had a program. We still don’t have anybody going in to 
provide any extra time or extra help because we had to put it in a set aside 
and so that money — $5 million dollars — is sitting unused, for a whole 
semester and kids aren’t getting extra time and extra help in Supplemental 
Services. (M. Canedo, personnel communication, February 5, 2003) 

Given the low response among students to supplemental services, the set aside 
requirements meant that large sums of money went unspent until late in the school 
year. This was the case across all districts and was not resolved by the U.S. 

Department of Education for the 2003-04 school year. According to the Associate 
Superintendent for Instruction in Buffalo, the amount of money set aside for 
supplemental services “seemed excessive based on the number of parents who usually 
request help for their kids” (district official, personal communication with G. 
Sunderman, February 6, 2003). These set aside requirements create perverse 
incentives for districts to use unspent money in instructionally ineffective ways. 

Since the money was not released until late in the school year, districts could not 
incorporate the money into a long-term reform strategy. For example, they were 
unlikely to use it to hire teachers or reduce class size since these kinds of decisions 
are rarely made at the end of a school year. 



The Title I, Part A alloeation ineludes funds for basie programs operated by the distriet and alloeations to 
loeal sehools. 
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Table 9: Title I District Allocations, Choice and Supplemental Service Set Aside, and 
Maximum Per Child Expenditure for Supplemental Services, in Current Dollars, FY 



2002 . 



District 


Title I Allocation, 
FY 2002 


Choice & Supplemental 
Service Set Aside 


Maximum Per Child Expenditure for 
Supplemental Services 


Mesa, AZ 


$ 10,894,656 


$2,178,931 


$905 


Washington, AZ 


3,761,149 


752,230 


802 


Fresno, CA 


37,360,593 


7,472,119 


991 


Los Angeles, CA 


303,003,900 


60,600,780 


1,037 


Chieago, IL 


216,575,070 


43,315,014 


1,322 


Buffalo, NY 


32,991,900 


6,598,380 


1,322 


New York, NY 


633,791,322 


126,758,264 


1,409 


Arlington, VA 


2,709,448 


541,890 


929 


Riehmond, VA 


9,979,219 


1,995,844 


1,002 


Atlanta, GA 


34,838,391 


6,967,678 


1,092 


DeKalb, GA 


15,057,883 


3,011,577 


1,046 



Source'. US Department of Education, ESEA Title I LEA Allocations - FY 2002. Retrieved from 
http://www.ed.gOv/offices/OUS/TitleILEAs/FY02allocations/ on 8-21-03 and 
www.ed.gov/about/overview^udget/titlei/fv02 on 12-1-03. 

Note'. All figures are rounded to the nearest dollar. For New York City, the Title I allocation and set aside is the sum 
of the allotments made to Bronx County, Kings County (Brooklyn), New York County (Manhattan), Queens County, 
Richmond County (Staten Island), and the per child expenditure represents an average across the five counties. The 
Los Angeles Board of Education approved $953.64 as the maximum per child expenditure (Los Angeles Board of 
Education Report No. 88 -02/03, March 11, 2003). 



District officials were concerned about the total cost of the program to the district as 
well as the per pupil cost. The law states that districts can reduce the Title I 
allocation to schools required to offer supplemental services by 15% to meet the set 
aside requirement. Since the supplemental services allocation is based on the total 
district Title I allocation, the maximum per child expenditure for supplemental 
services is greater than the per pupil allocation a school receives to provide Title I 
services. For example, in Fresno, supplemental services provided $991 per student 
whereas the highest poverty school received about $500 per student in Title I funds. 

In Richmond, the set aside was $1,002 per student while the Title I per pupil 
allocation was $600, and in Buffalo, the Title I per pupil allocation was $689 
compared to $1,322 set aside for supplemental services. The Title I per pupil 
allocation in Chicago, which uses a sliding scale based on a school’s level of poverty, 
ranged from $281 in a school with 50% poverty to $995 for a school with 92.82% 
poverty compared to $1,332 for supplemental services. Even though a school would 
not receive the money that went to pay the service provider, they would still be 
responsible for providing Title I services during the school day, even for a student 
receiving supplemental services. 

To control costs and manage administrative responsibilities, many of the districts applied 
to provide their own services (Mesa, Richmond, and Fresno were the exception to this). 
For example, in New York City district education offices and the central administration 
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accounted for 18 out of 47 providers. Some distriets actively promoted their own 
program. In Los Angeles, more students chose the district run program — Beyond the 
Bell — over the other 22 providers with 56.9% of students ehoosing the distriet program.^ 
Buffalo was not eligible to become a provider sinee it was designated a district in need of 
improvement. However, schools within such a district that have not been identified for 
improvement could be providers. Buffalo identified three sehools not in improvement 
and one parent eenter as service providers, whieh were approved by the state. In DeKalb 
County, where there was only one other loeal provider on the state-approved list, the 
district program provided parents with an option to the loeal provider. Providing the 
serviees themselves offered districts a number of advantages. Since costs were less for 
distriets, students received more serviees. It was also easier to traek district services and, 
since districts were still aecountable for student aehievement, their knowledge of the 
currieulum gave them a distinct advantage over outside providers. However, it runs 
eounter to the goals of earlier Title 1 legislation, whieh sought to eoordinate the delivery 
of instruction through schoolwide programs and to make services available in the 
schools. It also shifts responsibility for providing additional academie instruction and 
remediation away from loeal sehools to the district. 

Coordinating Supplemental Educational Services with the School Curriculum 

Many district officials were vague about how they would eoordinate the supplemental 
edueational serviees program with the sehool curriculum or how communication between 
the providers and the regular classroom instructors would be maintained. While still in 
the first year of implementation, some district officials already recognized that the 
interaction between supplemental services and ongoing programs and curriculum could 
pose a challenge. District officials from the Washington Elementary Sehool Distriet in 
Phoenix were concerned that supplemental services were at odds with district and school 
efforts to develop comprehensive programs for students in high poverty sehools. To meet 
the requirements of the law, the district developed a program that was offered for two 
hours on Saturday mornings. This was an expedient way for the distriet to meet the 
NCLB requirements, yet this centralized program was contrary to district efforts to 
integrate remediation and extra instructional time into the sehool day. Aecording to the 
superintendent: 

What we’ve been working on in our schools is to integrate the remediation 
and the extra time into the time you have them. And you have to 
eontinually do that, so when 1 talked about those qualities of effeetive 
sehools, one of our indieators is that you have embedded into your 
programming a way to give kids more time who need more time. But it’s 
not Saturday school. (T. Reale, personal eommunication with J. Kim & G. 
Sunderman, June 16, 2003) 

District officials in Buffalo believed that not only would it be harder for students to 
aceess serviees that were not loeated in the building where the student attended school, 
but that it would also be difficult to determine how well the provider’s instruction 



^ In 2002-03, 5,830 (56.9%) students chose the district program and 4,417 (43.1%) chose other providers. 
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coordinated with the regular currieulum and instruetion the student reeeived during the 
day. The distriet planned to develop a data traeking system based on student assessments 
to insure that supplemental serviees improved student aehievement. This data would be 
made available to classroom teachers. Richmond officials said providers were vague 
about how they would eommunicate with teachers. 

In other distriets, offieials believed that sinee providers were required to assess student 
needs and teach the state standards, students would receive instruction that coordinated 
with that reeeived in the elassroom. For example, the Beyond the Bell program offered 
by the distriet in Los Angeles is aligned with the state standards. To faeilitate 
eommunieation, providers in Los Angeles must give elassroom teaehers eopies of student 
learning plans that providers must develop for each student to guide serviees, and 
progress reports that require providers to eheek a box indieating whether the student is 
making satisfactory or unsatisfactory progress. There are no meehanisms for providers to 
reeeive information from teaehers. 
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CHALLENGES TO EVALUATING SUPPLEMENTAL EDUCATIONAL 

SERVICES 



The administrative and technical challenges to evaluating the effectiveness of 
supplemental educational services are formidable for local school districts. Local 
education officials must work with both parents and providers to develop an agreement 
that sets specific achievement goals for participating students. This agreement must 
describe plans for evaluating student progress, set a timeline for improving achievement, 
and provide regular information to parents and teachers about student progress (P. L. 
107-110, Sec. 1 1 16(e)(3)). According to the statute, the purpose of these activities is to 
ensure that supplemental educational services “are of high-quality, research-based, and 
specifically designed to increase the academic achievement of eligible children... and 
attain proficiency in meeting the state’s academic achievement standards” (P. L. 107-110, 
Sec. 1 1 16(e)(12)(C)(ii)). These requirements clearly place enormous burdens on districts 
to monitor the providers. However, districts receive no additional money to support the 
administration and evaluation of these programs. Since there are no additional resources 
for evaluations, the federal requirements create strong incentives for districts to minimize 
administrative and financial costs by using inexpensive and easy methods for evaluating 
supplemental education service providers. 

The ambiguous and lenient criteria for assessing the effectiveness of supplemental 
educational services combined with the short implementation timeline meant that most 
districts had rudimentary evaluation plans in place during the first year. For example, 
administrative staff in Los Angeles Unified responsible for administering supplemental 
educational services spent most of their time designing outreach brochures, notifying 
eligible students, processing applications for services, monitoring learning contracts, 
tracking attendance in programs, and running the district’s supplemental educational 
service program. The district did not conduct research on or evaluate the supplemental 
services programs, in part because they had been serving students for less than one full 
year. The district planned to partner with some state-approved providers, such as 
Voyager and Action Learning, to conduct pre- and post-tests for participating students in 
order to gather data on student learning during their time in the program (district official, 
personal communication with J. Wing, January 13, 2004). In Richmond, administrators 
planned to use similar methods. According to one Richmond administrator, since “the 
law requires a pre and post assessment,” most providers are conducting “a pre evaluation 
. . . [and] a post or exit evaluation” (district official, personal communication with G. 
Sunderman, May 27, 2003). Chicago administrators were considering evaluations that 
compared the performance of participating students and a similar group of Title I students 
who did not receive supplemental educational services (district official, personal 
communication with G. Sunderman, June 6, 2003). While it is important for districts to 
monitor both attendance and achievement, it is unlikely that these evaluation plans will 
provide a true measure of the effectiveness of the program. 

Sound and rigorous evaluations require careful planning, time, and adequate money. 
Indeed, elsewhere in NCLB, the term “scientifically-based research” is mentioned over 
100 times, and the statute gives preference to rigorous experimental studies. Yet, no such 
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requirement is plaeed on evaluations of supplemental serviees, making it extremely 
diffieult to evaluate their effeetiveness. It will be diffieult, if not impossible, to assess the 
eausal impaet of supplemental edueational serviees on student aehievement beeause 
students are not randomly assigned to these programs. The federal law and most state 
guidelines merely require distriets to look at the test seore gains of students before and 
after they reeeive supplemental edueational serviees.^ However, this kind of researeh 
design provides the weakest basis for identifying a eausal link between the intervention 
and outeome beeause it laeks an equivalent eomparison group. If researehers attempted 
to eompare outeomes for students reeeiving supplemental serviees with a similar group of 
non-partieipating students, the findings would be misleading for yet another reason: 
distriets must provide serviees first to the lowest-performing students from the poorest 
families. Thus, students reeeiving serviees will differ in unseen ways from non- 
partieipants even if adjustments are used to ensure that the two groups are eomparable. 
One way to overeome this methodologieal hurdle is to eonduet an experiment in whieh 
vouehers for supplemental serviees are randomly assigned to eligible students. This 
would be possible in many distriets where demand for free tutoring exeeeds the supply of 
available vouehers. However, it would also require that distriets reeeive approval from 
the U.S. Department of Edueation, sinee distriets must follow stringent rules for seleeting 
students for supplemental edueational serviees. 

There is a direet link between the need for quality evidenee of the effeetiveness of 
supplemental serviees and eivil rights, beeause the partieipants are overwhelming 
nonwhite and beeause there are major eosts assoeiated with these programs. In 
partieular, the loss of Title I funding may undereut the ability of high-poverty sehools to 
maintain a finaneial eommitment to a sehoolwide program that is intended to benefit all 
students. Researeh shows that a sehool must invest eonsiderable fiseal, administrative, 
and human resourees in faithfully implementing a eomprehensive sehool reform for five 
or more years before it produees maximum learning gains for students (Borman, Hewes, 
Overman, & Brown, 2003). If distriet budgets are redueed up to 20% to support 
supplemental edueation serviees, fewer high-poverty sehools will be able to make a 
sustained eommitment to a number of eomprehensive sehool reform programs that have a 
mueh stronger traek-reeord of improving ehildren’s reading and math skills than 
supplemental edueational serviees (Ameriean Institutes for Researeh, 1999; Borman & 
Hewes, 2002). 



^ For example, the Arizona State Department of Edueation has a seoring mbrie to evaluate the effeetiveness 
of supplemental edueational serviee providers. There are three eategories: poor quality (1-4 points), 
moderate quality (5-7 points), and high quality (8-10) points. Poor quality programs have “little or no 
evidenee to indieate that the program will be monitored for effeetiveness.” Moderate quality programs 
have “some evidenee provided that the program will be monitored for effeetiveness, however there are 
limited details.” Fligh quality programs have “substantial evidenee . . . that indieates that eaeh program will 
be monitored for effeetiveness.” Under this mbrie, providers do not have to perform rigorous experiments 
and there is no federal guideline stating the quality evidenee needed to determine whether a program is 
“high quality and researeh-based.” See “Supplemental Edueation Serviees Provider Applieations, Arizona 
Department of Edueation” http://www.ade.az.gov/asd/Titlel/SES/ 
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SUMMARY AND RECOMMENDATIONS 



The supplemental edueational services provision represents a major tenant of the No 
Child Left Behind Act. It assumes that competition will produce better educational 
opportunities for disadvantaged students than the public schools provide and that 
accountability for individual student achievement will improve the performance of low 
performing schools. This view has been articulated by federal policymakers (U.S. 
Department of Education, 2002a), who contend that the “supplemental educational 
services requirements of the law not only help to enhance student achievement but also 
provide an incentive for low-performing schools to improve. Schools that want to avoid 
losing students, not to mention restructuring, will have to do a better job” (p. 9). Yet few 
researchers have examined the impact of supplemental services on student achievement 
and school performance. There is no body of research that provides clear and consistent 
evidence documenting the effect of supplemental educational services on learning 
outcomes for low-income or minority students. This is ironic given the emphasis in the 
law on “scientifically based research.” 

Our research found that the potential for supplemental educational services to fragment 
Title I is real and not addressed in the law. Supplemental services revises the direction of 
earlier Title I legislation that encouraged programs designed to increase curricular and 
instructional integration. However, under NCLB supplemental services seriously disrupt 
other school reform efforts by diverting resources away from the most needy schools. 
There are no mechanisms to hold service providers accountable and no requirements in 
the law for serious and rigorous evaluations of the effectiveness of the programs. 
Supplemental services shift the focus from improving poorly performing schools to 
improving individual student achievement, but only for those requesting services. 
Combined with the loss of resources, it is unclear how this strategy will improve low 
performing schools. 

Our first year study highlights the enormous administrative burdens districts faced in 
implementing supplemental education services and the difficulties in assessing the effect 
of this policy on student achievement and Title I schools. NCLB supplemental 
educational services placed unprecedented responsibilities on school districts’ operating 
procedures and shifted responsibility for providing additional educational services from 
Title I schools to the district. Unlike intra-district choice, which many urban districts 
adopted prior to NCLB, virtually no public school district had used Title I money to 
enable the lowest-performing students to receive tutoring services from a for-profit 
company. To comply with the federal rules governing supplemental educational services, 
districts provided considerable administrative and managerial oversight of the 
development and implementation of the program, yet there were no additional 
administrative funds in Title I to cover these costs. None of the districts in this study had 
other than rudimentary plans to evaluate the effectiveness of the programs. 

Our analysis of student participation in supplemental educational programs during the 
first year found that demand for services was low, perhaps because the services were 
offered outside of regular school hours and away from eligible students’ neighborhood 
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schools. Our data also shows that this provision disproportionately impacts districts 
serving large numbers of low-ineome and minority students. 

While requests for supplemental services were low in the first year, the demand for free 
tutoring is likely to increase in most jurisdictions for two reasons: (1) more parents are 
ehoosing free tutoring over the transfer option, and (2) the number of students eligible to 
reeeive supplemental edueational serviees eontinues to grow sinee more sehools are 
failing to make adequate yearly progress and are subjeet to these federal sanctions. 
Meeting the growing demand for tutoring will plaee additional administrative burdens on 
distriet offioials and further divert resourees from sehools serving disadvantaged sehools. 
It is unclear how these schools will improve with less rather than more resourees. 

Policy Recommendations 

Whether supplemental educational services ultimately help or hurt the aehievement of 
disadvantaged students is an empirieal question that has deep implications for educational 
equity. For now, better evidenee is needed before publie dollars are used to support 
supplemental edueational serviee providers on a large-seale. Given what is not known 
about supplemental serviees, sealing up an educational policy with potentially limited 
benefits for student aehievement and serious fiseal eonsequenees for publie school 
districts is not good public policy. We make the following reeommendations. 

• The federal government should terminate the supplemental educational 
services program as a mandated sanction for poorly performing schools. 

Until there is solid empirieal evidenee of its effeetiveness, the provision of 
supplemental edueational serviees should be limited to a series of field trials 
implemented on a small seale and in a way that does not disrupt other sehool 
reform efforts with demonstrated effeetiveness. 

• The federal government should fund randomized experiments that assess the 
effectiveness of supplemental educational services in improving student 
achievement. In keeping with the legislation’s foeus on “soientifieally-based 
researeh,” whieh give preferenee to randomized experiments, supplemental 
edueational serviees should be subjected to the same rigorous standards of 
evidence required of other edueational interventions. Evaluations should also 
assess the eosts and benefits of administrating supplemental edueational serviees 
and researeh should be eondueted to identify the best ways to design a program 
that will insure maximum educational benefit. 

• Policymakers need to revisit the supplemental educational services provision 
in light of the earlier consensus on the direction of Title I services. The idea of 
meehanisms that would integrate the Title I program with the regular sehool 
eurrieulum and give schools greater flexibility in how to use their federal Title I 
resourees has merits and should be strengthened. 

• If the federal government continues requiring supplemental educational 
services — ^which we do not recommend — the set-aside should he forward 
funded. By forward funding supplemental serviees, resourees would be 
available for other sehool reform initiatives at the beginning of the sehool year 



33 




rather than at the end. As we have learned from general funding for Title I, 
money that eomes late in the sehool year has little value and diseourages long 
term planning. It is essential that the federal government find ways to fund 
supplemental edueational services that do not rely on diverting resources away 
from the most disadvantaged schools. Punishing schools that serve our most 
vulnerable students by removing resources is unlikely to help them improve. 
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APPENDIX 



Appendix 1; Number of Title I Schools Identified as Needing Improvement for 2002-03 
and 2003-04. 



State 


2002-03 (Last Updated, June 2003) 


2003-04 (Last Updated, December 9, 2003) 


Arizona 

(Source) 


399 

Personal Communication-Title I 


244 

http://www.ade.az.gov/nrofile/Dublicview/avDschoollist.asD. 


California 

(Source) 


815 

http://www.cde.ca.gov/iasa/titleone/pi/ 


1,205 

http://www.cde.ca.gOv/ayp/2003/titleone/titleI_layout.htm 


Illinois 

(Source) 


527 

httD://www.isbe.net/research/renorts.htm#Statistics 


581 

http://www.isbe.net/research/pdfs/2003_StateReport_E.pdf 


New York 
(Source) 


434 

http://www.emsc.nysed.gov/deputy/nclb/nclbhome.htm 


517 

http://www.emsc.nysed.gov/deputy/nclb/nclbhome.htm 


Georgia 

(Source) 


436 

http ://www. doe .k 1 2 .ga.us/ support/plan/ nclb .asp 


846 

http://www.doe.kl2.ga.us/support/plan/nclb.asp 


Virginia 

(Source) 


34 

http ://www.pen.k 1 2 . va.us/VDOE/ src/vasrc-title 1 .pdf 


43 

http ://www.pen.k 1 2 . va.us/VDOE/ src/vasrc-title 1 .pdf 



For Table 3, the number of Title I schools identified as needing improvement in each 
district in the study was extracted from state data files by filtering state lists according to 
each district’s unique identifier code. For Los Angeles, the state identified 102 program 
improvement schools for 2002-03. However, the district included 106 schools when 
deciding which schools must offer students the option to transfer or the opportunity to 
receive supplemental services. This included schools that were later removed from the 
list of program improvement schools. For Table 4, a similar method was used to identify 
the number of schools that were required to offer supplemental services. The number of 
schools required to offer supplemental services is a subset of the number of schools 
identified for improvement. We computed the total enrollment and number of eligible 
students for the schools required to offer supplemental services to obtain the figures in 
Table 5. We also disaggregated these numbers by race/ethnicity to compute the numbers 
used to create Figure 1. 
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Appendix 2: Data Sources for Tables 6 and 7. 

Data for the number of students eligible for supplemental services was obtained by 
summing the number of students eligible for free and reduced price lunch in all schools 
that failed to make adequate yearly progress for three consecutive years and thus were in 
their second year of school improvement. We verified this information by cross checking 
our numbers with multiple sources of data, including information provided by district 
officials (in direct communication) and from newspaper articles. Where data from 
multiple sources conflicted, the authors cited the data deemed most reliable. Data 
obtained directly from the districts or arrived at by using data provided by the districts 
was given more weight than data provided by media sources. For Los Angeles, New 
York, and Atlanta, the data cited was provided directly by district officials. For the other 
districts, the reported data was calculated using the number of students eligible for free 
and reduced price lunch enrolled in eligible schools. The following discrepancies are 
noted. 

The Fresno Bee reported that approximately 17,000 students were eligible in Fresno.^ 
Summing the enrollment data for all schools in Fresno that failed to make adequate 
yearly progress for three consecutive years and thus were in their second year of school 
improvement produced a figure of 16,831 eligible students. 

The Los Angeles Unified School District reported that 164,434 students were eligible in 

o n 

Los Angeles. The Los Angeles Times also reported this number. Summing the number 
of students eligible for free or reduced price lunch in all schools in Los Angeles that 
failed to make adequate yearly progress for three consecutive years and that were in their 
second year of school improvement produced a figure of 165,257 eligible students. 

The Chicago Sun-Times reported that 17,000 students were eligible in Chicago. 

Summing the number of students eligible for free or reduced price lunch in all schools in 
Chicago that failed to make adequate yearly progress for three consecutive years and 
were in their second year of school improvement produces a figure of 17,455 eligible 
students. 

New York City Department of Education reported that 243,249 students were eligible in 
New York City. '' The New York Times confirmed this number. Summing the number 
of students eligible for free or reduced price lunch in all schools in New York City that 
failed to make adequate yearly progress for three consecutive years and thus were in their 
second year of school improvement produces a figure of 224,644 eligible students. 



^ Ellis, A.D. (2002, November 30). Few take part in Fresno tutoring program; Many more eligible to 
take part in No Child Left Behind. The Fresno Bee, p.Al . 

* B. Robinson (personal eommunieation, January 13, 2004). 

^ Helfland, D. (2003, September 23). L.A. sehools promote free tutoring for some students. The Los 
Angeles Times, p. 4 

Rossi, R. (2003, Mareh 19). Chieago students to get private tutors. The Chicago Sun-Times, p. 16. 

*’ M. MeManus (personal eommunieation, Deeember 16, 2003) 

Gross, J. (2003, August 29). Free tutoring reaehes only a fraetion of students. The New York Times, p. 
Al. 
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Atlanta Public Schools officials reported that 4,249 students were eligible in Atlanta. 

This number was eonfirmed on the Atlanta Publie Schools’ web site.'"^ Summing the 
number of students eligible for free or reduced price luneh in all schools in Atlanta that 
failed to make adequate yearly progress for three consecutive years and thus were in their 
seeond year of sehool improvement produces a figure of 13,448 eligible students. The 
Atlanta Journal Constitution reported that 13,005 students were eligible in Atlanta.^^ 

The Atlanta Journal Constitution reported that 10,571 students were eligible in DeKalb 
County.'^ Summing the number of students eligible for free or reduced price luneh in all 
sehools in DeKalb that failed to make adequate yearly progress for three consecutive 
years and thus were in their second year of school improvement produees a figure of 
9,355 eligible students. 

Data for the number of requests for supplemental educational services was obtained from 

interviews and personal communication with district staff Mesa Public Schools reported 

that 2 students requested services.'^ Washington Elementary School District reported 

that 20 students requested services. Fresno Unified School District reported that 234 

students requested serviees.'^ The Fresno Bee reported that 227 students requested 

services in Fresno.^° For the Fos Angeles Unified Sehool District, we caleulated that 

2 1 

11,518 students requested supplemental serviees using data provided by the distriet. 

The Los Angeles Times reported that approximately 10,000 students requested services in 
Fos Angeles. The Los Angeles Daily News reported that approximately 12,000 students 
requested services in Fos Angeles^^. Chicago Public Schools reported that 1,400 students 
requested services. Our finding that 849 students requested serviees in Buffalo was 
contained in eorrespondence received from the Buffalo Public Schools. New York City 
Department of Education reported that 30,349 students requested services. Riehmond 
Public Schools reported that 600 students requested services. Atlanta Public Schools 



S. Coleman (personal communication, December 5, 2003) 

Talk Up Atlanta: Read the Facts about No Child Left Behind in APS. 

http://www.atlanta.kl2.ga.us/news/goodnews/121703a.html 
Ghezzi, P. (2003, April 4). Slow start for ‘No Child’ law. The Atlanta Journal-Constitution, p. lA. 
Ghezzi, P. (2003, April 4). Slow start for ‘No Child’ law. The Atlanta Journal-Constitution, p. lA. 
J. O’Reilly (personal communication, June 17, 2003) 

** J. Sullivan (personal communication, June 16, 2003) 

B. Bengel (personal communication, July 31, 2003) 

Ellis, A.D. (2002, November 30). Few take part in Fresno tutoring program; Many more eligible to 
take part in No Child Left Behind. The Fresno Bee, p.Al . 

B. Robinson (personal communication, January 13, 2004) 

Helfland, D. (2003, September 23). L.A. schools promote free tutoring for some students. The Los 
Angeles Times, p. 4. 

Gao, Ft. (2003, September 19). Thousands ignore chance for kids to get free tutoring. The Los 
Angeles Daily News. 

E. Collier (Personal communication, June 27, 2003) 

A. Turley letter to New York State Education Department (September 5, 2003) 

M. McManus (personal communication, December 16, 2003) 

E. Scott (personal communication. May 27, 2003) 
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reported that 2,380 students requested serviees. DeKalb County Publie Sehools 
reported that 575 students requested serviees. 

The pereentage of student requesting supplemental serviees were ealeulated using the 
number of students requesting serviees and the number of students eligible for serviees. 
Both the Assoeiated Press and the New York Times reported that 12.5 pereent of the 
eligible students in New York City reeeived supplemental serviees. 

Data for the number of students who reeeived supplemental serviees was obtained from 
personal eommunieation with distriet staff. Mesa Publie Sehools reported that 2 students 
reeeived serviees. Washington Elementary Sehool Distriet reported that 20 students 
reeeived serviees. Fresno Unified Sehool Distriet reported that 36 students reeeived 
serviees. Los Angeles Unified Sehool Distriet reported that 10,247 students 
partieipated. Chieago Publie Sehools reported that 850 students reeeived serviees. 

Our finding that 573 students reeeived serviees in Buffalo was eontained in 
eorrespondenee reeeived from the Buffalo Publie Sehools. New York City Department 
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of Edueation reported that 30,349 students reeeived serviees. The New York Times 
initially reported that 30,333 students requested serviees and later reported that 30,359 
students reeeived serviees"^° in New York City. Richmond Public Schools reported that 
122 students received services."^' Atlanta Public Schools reported that 2,380 students 
received services. The Atlanta Journal-Constitution reported that 373 students had 
received services in Atlanta."^^ DeKalb County Public Schools reported that 575 students 
received services."^"^ The Atlanta- Journal Constitution confirmed that 575 students 
received services in Atlanta."^^ The percentage of students receiving services were 
calculated using the number of students receiving services and the number of students 
eligible for services. 

Note: Arlington, VA had no improvement schools. 



S. Coleman (personal communication, December 5, 2003) 

P. Speaks (personal communication, February 14, 2003) 

More students have signed up for free tutoring, officials say. (2003, November 6). The Associated 
Press. 

Gross, J. (2003, November 6). Free tutoring is reaching more students in the system. The New York 
Times, p. B8. 

J. O’Reilly (personal communication, June 17, 2003) 

J. Sullivan (personal communication, June 16, 2003) 

B. Bengel (personal communication, July 31, 2003) 

B. Robinson (personal communication, January 13, 2004) 

E. Collier (Personal communication, June 27, 2003) 

A. Turley letter to New York State Education Department (September 5, 2003) 

M. McManus (personal communication, December 16, 2003) 

Gross, J. (2003, August 29). Free tutoring reaches only a fraction of students. The New York Times, 
p.Al. 

Gross, J. (2003, November 6). Free tutoring is reaching more students in the system. The New York 
Times, p. B8. 

E. Scott (personal communication. May 27, 2003) 

S. Coleman (personal communication, December 5, 2003) 

Ghezzi, P. (2003, April 4). Slow start for ‘No Child’ law. The Atlanta Journal-Constitution, p. lA. 
P. Speaks (personal communication, February 14, 2003) 

Ghezzi, P. (2003, April 4). Slow start for ‘No Child’ law. The Atlanta Journal-Constitution, p. lA. 
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